Lecture 8 : Source Coding Theorem , Huffman coding

Fix a random variable X having as its possible values entries of a set X . Let Σ∗ denote the set of strings whose characters are taken from an alphabet Σ. In the case Σ has two elements, we can interpret elements of Σ∗ as bit strings, for example. Let Y denote a set of target symbols and C denote a code, a function of the form X → Y∗. The extension of a symbol code, a code of the form C : X → Y∗, is the function C : X ∗ → Y∗ defined by C(x1x2 . . . xn) = C(x1)C(x2) . . . C(xn), n = 0, 1, . . . , x1, x2, . . . , xn ∈ X.